Goodware Hash Sets

    Published: 2025-01-02. Last Updated: 2025-01-02 15:21:40 UTC
    by Xavier Mertens (Version: 1)
    0 comment(s)

    In the cybersecurity landscape, we all need hashes! A hash is the result of applying a special mathematical function (a “hash function”) that transforms an input (such as a file or a piece of text) into a fixed-size string or number. This output, often called a “hash value,” “digest,” or “checksum,” uniquely represents the original data. In the context of this diary, hashes are commonly used for data integrity checks. There are plenty of them (MD5, SHA-1, SHA-2, SHA-256, …), SHA256 being the most popular for a while because older like MD5 are considered as broken because researchers have demonstrated practical collision attacks.

    Hashes are a nice way to identify malware samples, payload, or any type of suspicious files (I usually share the hash of the malware analyzed in my diaries). In your threat-hunting process, you can search for interesting files across your infrastructure via sets of malware hashes. Some of them are freely available like on Malware Bazaar[1].

    But, other sets of hashes are also interesting when they contain hashes for safe files. The approach is the same: Instead of searching for malicious files, you verify that files on your hosts are good.

    Exacorn has released an interesting ZIP archive[2] with “good ware” (as opposed to “malware”). The file (2GB) provides 12M hashes and filenames:

    Pay attention that some files might be flagged by some antivirus solutions. For example, I searched for "putty.exe" in the file. One of the returned hashes is: 6CDBE5323E1DEC7102D86C60458D6C7465807E80516D63F2EE509625C1DF2416[3].

    It’s a perfect opportunity to remind you that other projects exist. The ones that I use regularly:

    • The National Software Reference Library (NSRL) project[4]
    • The CIRCL.lu Hash Lookup API[5]
    • Hashsets.com [6] (not 100% free)

    I like the second one because it includes the NSRL lists and can be used in an automated way.

    We love hashes!

    [1] https://bazaar.abuse.ch/export/
    [2] https://www.hexacorn.com/blog/2024/12/31/clean-hash-set-12m-rows/
    [3] https://www.virustotal.com/gui/file/6cdbe5323e1dec7102d86c60458d6c7465807e80516d63f2ee509625c1df2416
    [4] https://www.nist.gov/itl/ssd/software-quality-group/national-software-reference-library-nsrl/about-nsrl
    [5] https://www.circl.lu/services/hashlookup/
    [6] https://www.hashsets.com
     

    Xavier Mertens (@xme)
    Xameco
    Senior ISC Handler - Freelance Cyber Security Consultant
    PGP Key

    0 comment(s)

      Comments


      Diary Archives